Emotion, mood and personality inference in real-time environments

ABSTRACT

Methods and systems monitor communications between users and analyze the communications to simultaneously determine, for a current time period, mental state variables of one of the users. Such mental state variables include the emotion of the user, the mood of the user, and the personality of the user. Additionally, such methods aggregate the emotion, the mood, and the personality using a hierarchical probabilistic graphical model that determines the highest probability path through a directed probabilistic graph to infer the mental state of the user. The directed probabilistic graph maintains a single state for the personality for the time period, and maintains multiple states for the emotion and the mood for the time period. These methods and systems output the mental state of the user.

BACKGROUND

Systems and methods herein generally relate to using specializedmachines to monitor communications between users, and processes thatoutput and constantly revise the mental state of one or more of theusers.

The issue of customer modeling (e.g., understanding who a customer is)is fundamental to any notion of personalization and is an issueparticularly with Virtual Agent processes. For successful communication,a useful component of the customer to model is their mental state: theirpersonality, mood, and emotions.

SUMMARY

Exemplary methods herein automatically monitor text and/or speechcommunications between users using a specialized language processor, andautomatically analyze the communications using the specialized languageprocessor to simultaneously determine, for a current time period, mentalstate variables of a user. These mental state variables can include, forexample, the emotion of the user, the mood of the user and thepersonality of the user. The method then automatically aggregates theemotion, mood, and personality using a hierarchical probabilisticgraphical model to determine the highest probability path through adirected probabilistic graph to infer the mental state of the user.Using the specialized language processor, the method outputs the mentalstate of the user from the specialized language processor by displayingthe emotion, mood, and personality on the graphic user interface of theprocessor, or outputting the mental state to a different process.

The directed probabilistic graph maintains a single state forpersonality for the time period, and maintains multiple states for theemotion and the mood for the time period. Therefore, this directedprobabilistic graph has a single personality node, multiple mood nodes,multiple emotion nodes, and multiple evidence nodes. The directedprobabilistic graph has edges connecting the personality node, the moodnodes, the emotion nodes, and the evidence nodes; and the edgesthemselves have probability values. The method processes a path throughthe directed probabilistic graph, and the probability of the path isformed from an aggregation of the probabilities of the edges of theseries of adjacent nodes. The highest probability path has anaggregation of the probabilities of the edges that is higher than allother possible paths through the directed probabilistic graph.

Each of the mood nodes, the emotion nodes, and the evidence nodes arefor a different time portion of the time period. The evidence nodes caninclude different dialogue variables used by the personality node, themood nodes, and the emotion nodes. The emotion nodes can be, forexample, happy-for, satisfaction, anger, or distressed states; the moodnodes can be, for example, positive, neutral, or negative, and thepersonality nodes can be, for example, neuroticism, extraversion,openness to experience, agreeableness, or conscientiousness.

Exemplary systems herein include a specialized language processor andany form of interface (e.g., a graphic user interface) connected to thespecialized language processor. The specialized language processorautomatically monitors text communications between users, and thespecialized language processor automatically analyzes the textcommunications to simultaneously determine, for a current time period,the mental state variables of a user. These mental state variablesinclude the emotion, personality and mood of the user. The specializedlanguage processor automatically aggregates the emotion, mood, andpersonality using a hierarchical probabilistic graphical model thatdetermines the highest probability path through the graph to infer themental state of the user. The graphic user interface then outputs themental state of the user from the specialized language processor, forexample by displaying the emotion, mood, and personality status or byoutputting the mental state to a different process.

The directed probabilistic graph maintains a single state forpersonality for the time period, and maintains multiple states for theemotion and the mood during the same time period. Therefore, thedirected probabilistic graph described above includes a singlepersonality node, multiple mood nodes, multiple emotion nodes, andmultiple evidence nodes. The edges of the directed probabilistic graphconnect the personality node, the mood nodes, the emotion nodes, and theevidence nodes and the edges contain probability values. A path throughthe directed probabilistic graph is made of a series of adjacent nodes,and the probability of the path is determined by an aggregation of theprobabilities of the edges of the series of adjacent nodes. Thus, thehighest probability path has an aggregation of the probabilities of theedges that is higher than all other possible paths through the graph.

Furthermore, each of the mood nodes, emotion nodes, and evidence nodesis for a different time portion of the time period and the evidencenodes include different dialogue variables used by the personality node,mood nodes, and emotion nodes. The emotion variables include, forexample, happy-for-satisfaction, anger, or distress, mood variablesinclude, for example, positive, neutral, or negative, and thepersonality variables include, for example neuroticism, extraversion,and openness to experience, agreeableness, or conscientiousness. Theseand other features are described in, or are apparent from, the followingdetailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

Various exemplary systems and methods are described in detail below,with reference to the attached drawing figures, in which:

FIG. 1 is a table diagram illustrating methods herein;

FIG. 2 is a hierarchical probabilistic graphical model of variousmethods herein;

FIG. 3 is a schematic diagram illustrating output produced by systemsand methods herein;

FIG. 4 is a schematic diagram illustrating output produced by systemsand methods herein;

FIG. 5 is a schematic diagram illustrating output produced by systemsand methods herein;

FIG. 6 is a schematic diagram illustrating a system herein;

FIG. 7 is a flowchart diagram illustrating methods herein;

FIG. 8 is a schematic diagram illustrating a system herein; and

FIG. 9 is a schematic diagram illustrating devices herein.

DETAILED DESCRIPTION

As mentioned above, one advance of customer modeling is to understandwho the individual customer is; and the systems and methods hereinprovide a probabilistic approach to tracking the mental state of thecustomer at each of three levels (e.g., personality, mood and emotion)during a sequential set of turns that compose a conversation. This canbe done on a number of levels from external (e.g., the products/servicesthat they own and use) through personal demographics (e.g., location,age, gender) to internal mental states and beliefs (e.g., personality,sentiment).

As humans make an impression on one another, they wish to automaticallydetermine the personality of a customer. Personality traits aregenerally considered temporally stable, and thus this disclosure'smodeling ability is enriched by the acquisition of more data over time.

Further, an individual's mood and emotions will cloud the ability todetermine personality. Indeed, implicit personality theory considersthat there are many factors that affect the impressions one forms ofpeople, including mood. The systems and methods described herein providean approach that enables a user to statistically infer mental states atseveral levels of temporal stability. For purposes herein the “mentalstate” includes three distinct, yet connected levels: personality, moodand emotion. More specifically, the systems and methods herein infer thethree levels by hierarchically connecting models together in a coherentprobabilistic graphical model (PGM). The systems and methods hereinprovide a formal PGM that infers an individual's mental states (thelatent variables) at the personality, mood and emotion levels fromevidence (observed variables).

The systems and methods apply this PGM in a temporally dynamicsituation: namely conversational dialogue data. In practice, this datacould be drawn from a direct 1-on-1 dialogue (for example a web chat) oran asynchronous series of communications via social media (e.g., a forumthread). The important considerations with the data are such that; thereis a conversational partner(s) who provide external utterances to theindividual which could affect their mental state. The communication isrelatively time-bound such that it is realistic to infer a connectionbetween short-term emotional states.

Firstly, the systems and methods herein present a dynamic andhierarchical mental state model. One broad concept herein is toaggregate in a hierarchical probabilistic framework several approachesto mental state modeling and specifying the necessary conditionaldependencies between them. In the second part, the systems and methodsaddress the inference procedure associated with the model. Thus, asshown in the table 100 in FIG. 1, when considering personality, mood,and emotion various observations can be made. The description of auser's personality is typically considered as set of characteristicspossessed by a person that uniquely influences their behavior, moderatedby context. While many models can be used to determine personality withsystems and methods herein, one exemplary model is the Five Factor Model(FFM) of neuroticism, extraversion, openness to experience,agreeableness, and conscientiousness. Further, as shown in FIG. 1, forpurposes herein, the personality is considered to be stable with respectto temporal stability.

As also shown in FIG. 1, the description of a user's mood is a lessspecific and less intense subjective state of mind than emotions, thoughtypically more prolonged. Various models calculate mood with systems andmethods herein using three states positive/good, neutral, andnegative/bad. Further, as shown in FIG. 1, for purposes herein, the moodis considered to be short-midterm with respect to temporal stability.

FIG. 1 also shows that the description of a user's emotion is asubjective state of mind experienced by an individual, most oftentriggered by a specific event, which expresses itself in manypsycho-physiological ways. While many models can be used to determinepersonality with systems and methods herein, one exemplary model is theOCC model (Ortony, Clore and Collins,) defines 22 categories includinghappy-for, satisfaction, anger, distress. These can be mapped to 6high-order, universally recognized categories.

FIG. 2 illustrates the Dynamic and Hierarchical Mental State Model(DHMSM) 102 that the systems and methods provide in order to inferhidden mental state variables from dialogue data with Tturns ofutterances exchanged. In the model, dialogue variables are defined andreferred to as evidence. In this directed probabilistic graphical model,the observed variables are shaded and the hidden variables that areinferred are blanked.

As shown in FIG. 2, the permanent aspect of the personality of aconsidered user u is defined by a random variable λ^(u) drawn from, forexample, a Dirichlet distribution of parameter θ_(λ)∈

⁵. The dimensionality of size 5 of this variable, in this instance, aimsat modeling the so-called Five Factor Model of personality; neuroticism,extraversion, openness, agreeableness and conscientiousness referencedin FIG. 1; however, those skilled in the art would understand thatdifferent personality models could have different dimensionalities. Fora given user, though the degree of personality can vary with context,this variable is considered stable through the time of the interactionsof a given dialog.

Further, in FIG. 2, the mood β^(u) is modeled as a temporal type ofrandom variable representing the 3 exemplary cardinal moods; good,neutral and bad drawn from a multinomial distribution of parameterθ_(β)∈

³. This example uses a Markovian dependency between the overallpersonality variable λ^(u), a mood state at time t, β_(t) ^(u) and theprevious mood state at time t−1, β_(t−1) ^(u); although those skilled inthe art would understand that other known dependencies could be usedwith systems and methods herein. Thus, FIG. 2 illustrates that that moodchange is a phenomena conditioned by the past state of the variable butalso the personality type of the user. For example, high scorers ofneuroticism are more prone to negative moods and dramatic mood changesthan low scorers.

Then, as further shown in FIG. 2, the instantaneous emotion state α^(u)is a random variable drawn from a multinomial distribution of parameterθ_(α)∈

⁶ that aims at representing six exemplary high level emotional states;hope, fear, relief, satisfaction, joy and distress (while those skilledin the art would understand that other emotional states could be used bysystems and methods herein). Note that this could be extended to thefull OCC model by using the parameter distribution θ_(α)∈

²².

Additionally, in FIG. 2, e^(u) and e^(m) are respectively the evidencesproduced by the user u and the dialogue partner m. The evidence issomething that is written by a user, something that is said by a user,something in the voice signal of the user, etc. In this model, thepartner could be either a human interlocutor, or a system producingautomated responses. These evidential variables can be decomposed tolinguistic and statistical features including, but not limited to, thosesuch as; bag-of-word ngrams, part-of-speech tags and parse treefeatures, dialogue acts associated to each utterance, message length,time of response, frequency of multiple utterances in the same turn,number of messages of a given turn, etc.

In this example, the partner m holds the initiative of the dialogue suchthat each observed utterance produced by the user u is statisticallyconditioned by both the current instantaneous emotional state and thelast utterance of the dialogue partner e_(t) ^(m). See equation (1)below;

p(λ^(u),β_(1:T) ^(u),α_(1:T) ^(u) ,e _(1:T) ^(u) ,e _(1:T)^(m))=p(λ^(u))p(β₁ ^(u)|λ^(u))Π_(t=2) ^(T) p(β_(t) ^(u)|λ^(u),β_(t−1)^(u))Π_(t=1) ^(T) p(α_(t) ^(u)|β_(t) ^(u))Π_(t=1) ^(T) p(e _(t) ^(u) |e_(t) ^(m),α_(t) ^(u))p(e _(t) ^(m)).

Equation (1) (above) defines the closed form expression of the jointprobability of the graphical model of the systems and method herein.During the inference phase (see below) the parameters of the mentalstate model _({λ) _(u) _(,β) _(1:T) _(u) _(,α) _(1:T) _(u) _(}) areinferred with respect to the observed variables _({e) _(1:T) _(u) _(,e)_(1:T) _(m) _(}) (the evidence utterances and derived features) asexpressed in Equation (2) (see below), which defines the correspondingmaximum a posteriori query:

${\underset{\lambda^{u},\beta_{1\text{:}T}^{u},\alpha_{1\text{:}T}^{u}}{argmax}\; {p\left( {\lambda^{u},\beta_{1\text{:}T}^{u},{\alpha_{1\text{:}T}^{u}e_{1\text{:}T}^{u}},e_{1\text{:}T}^{m}} \right)}} = {\frac{{p\left( {e_{1\text{:}T}^{u},{e_{1\text{:}T}^{m}\lambda^{u}},\beta_{1\text{:}T}^{u},\alpha_{1\text{:}T}^{u}} \right)}{p\left( {\lambda^{u},\beta_{1\text{:}T}^{u},\alpha_{1\text{:}T}^{u}} \right)}}{p\left( {e_{1\text{:}T}^{u},e_{1\text{:}T}^{m}} \right)} \propto {{p\left( {e_{1\text{:}T}^{u},{e_{1\text{:}T}^{m}\lambda^{u}},\beta_{1\text{:}T}^{u},\alpha_{1\text{:}T}^{u}} \right)}{{p\left( {\lambda^{u},\beta_{1\text{:}T}^{u},\alpha_{1\text{:}T}^{u}} \right)}.}}}$

According to Equation (2), two situations can be considered. Startingwith a uniform, i.e., non-informative, prior over the marginaldistribution of the parameters p(λ^(u),β_(1:T) ^(u),α_(1:T) ^(u)).Alternatively, it can be assumed that a given prior distribution ofthese variables, for a specific user u, has already been inferred in aprevious dialogue session analysis or by any other means. Concerning thesecond part of the equation, the likelihood of the evidence with respectto the model's parameters p(e_(1:T) ^(u),e_(1:T) ^(m)|λ^(u),β_(1:T)^(u),α_(1:T) ^(u)) will be maximized by, for example, Monte Carlo MarkovChain sampling.

Thus, with the systems and methods herein, the task of inferring theparameters of the model from data is also called learning. In thiscontext, one can assume the existence of an annotated corpus ofdialogues where each level of the hierarchical model, α^(u) and β^(u),is informed at each turn. Concerning λ^(u), the variable is informed atthe level of each dialogue. In fact, the computational challenge inlatent variable modeling is to compute the posterior distribution of thelatent variables conditioned by available observations. Except inrudimentary models, exact posterior inference is known to be intractableand practical data analysis relies on efficient approximatealternatives.

As noted above, in one example the systems and methods can apply aMarkov Chain Monte Carlo (MCMC) as a general technique for parameterinference in graphical models. MCMC sampling is the most widely usedmethod of approximate inference. The idea behind MCMC is to approximatea distribution by forming an empirical estimate from samples. One canconstruct a Markov chain with the appropriate stationary distribution,and collect the samples from a chain that has converged. One exemplaryprocess used with the systems and methods herein of a MCMC process isthe Gibbs sampler, in which the Markov chain is defined by iterativelysampling, in a sweep manner, each variable conditional on previouslysampled values of the other variables. This is a form of theMetropolis-Hastings process, and thus yields a chain with the desiredstationary distribution. In this modeling mentioned in the previousparagraph, every variable is sampled according to each correspondingdistribution.

Finally, the proposed generative model can also be used in a priorknowledge equipped setting. Indeed, assuming a customer can beidentified through-out a series of dialogues, it will be possible to setan informative prior on the λ^(u) parameter of the model.

Humans are very good at forming impressions of one another's personalityand mood. However, in a text-based chat dialogue, there is minimalextra-linguistic information (e.g., voice, facial expressions, bodylanguage) upon which one can form an impression. One embodiment of thesystems and methods is as part of an interface to support a human agentin understanding who their customer is (in this example case, in termsof personality and mood).

This is shown in an example presented in FIGS. 3-5. In FIG. 3, anexample chat interface (item 116) is shown between users, in this casean agent and a customer. The chat interface also includes additionalboxes depicting the CRM (customer resource management) data as shown initem 118 and the customer information shown in item 120.

More specifically, in the dialogue shown in item 104, the agent statesto the customer “Hello and welcome to our customer service line. Whatcan I help you with today.” These statements can be manually generatedby a human agent or automatically generated by a virtual (computergenerated) agent. The customer responds The internet doesn't work on myphone.” From the vagueness of this statement, the related methodsdetermine that the customer has an expertise level of “novice” as shownin item 110.

With the emotional state tracking ability of this system and method, theassessment of the customer will change over time. For example, in theinteraction shown in FIG. 3 there is not enough information in theopening turn of the dialogue (as seen in the dialogue text in item 104)to make any determination of mood or personality (nor is there any priorknowledge of the personality of this customer). However, again, thecustomer information section in item 120, is able to determine using thedialogue in item 104, that in this example, the customer is a novice inregards to the technology, as shown by item 110.

However, as the dialogue progresses, the systems and methods track themental state of the customer and update the reporting. This can be seenin FIG. 4, where the additional dialogue between the two users (item106) has enabled the systems and methods to determine the mood andpersonality of the customer, as seen in items 112 and 114. Morespecifically, the virtual or real agent states: “I'm very sorry to hearthat the internet doesn't work, let me try to help you with that”; andthe customer responds with a very negative statement: The wifi works asexpected but the 4G service I'm paying for does not. The config is as Iwas told it should be. There is no error in the proxy or IP address so Ineed you to fix this.” Because of the very negative statement of thecustomer, the systems and methods herein automatically determine thatthe customer's mood is “negative” and display the same in the customerinformation section 120, shown in FIG. 4 as item 112. In addition, thesystems and methods herein analyze the customer's response 106 andautomatically determine that the customer's personality is “direct,immediate” as shown by item 114 in FIG. 4.

As shown in FIG. 5, the third turn in the dialogue 108 includes thevirtual or real agent stating “Okay, give me a second. Trythis—Settings>Data>Network . . . ” and the customer happilyresponds:“OK, that's seems to have worked. Thank you so much for thehelp.” Because of the positive statement of the customer in 108, thesystems and methods herein automatically determine that the customer'smood is “positive” and display the same in the customer informationsection 120, shown in FIG. 4 as item 122.

Thus, as shown in the example in FIGS. 3-5, the models used by thesystems and methods herein are able to determine that though thepersonality 114 is stable, the mood of the customer changes from 112 to122 with the resolution of the dialogue as can be seen in FIGS. 4 and 5.The systems and methods again use the additional dialogue between thetwo users, shown in item 108, to update the mental state variables ofthe customer. In addition to having solved the issue of the customer,the inferred change in their mood from negative to positive, as shown initem 112 and 122, can be seen as a secondary successful outcome of theinteraction.

Customer modeling is a component of various automation projects. Asshown in FIG. 6, using the systems and methods described herein, thecustomer models are an observer of the dialogue between the humancustomer 140 and the agent (VA) 142. Note that agent 142 is notnecessarily a virtual agent, and could be a human agent who can interactwith the system for other reasons. In one example, the virtual agentcould be the combination of 130, 132 and 138. Also, element 134 could beincluded in the virtual agent in some situations. The systems andmethods herein replicate human impression formation for the agent 142,allowing the agent 142 to adapt to an increasing understanding of thepersonality and mood of the customer 140. By knowing the mental stateparameters of the customer 140, the systems and methods herein bias theselection of the dialogue act in the dialogue manager 132. Similarly, tothe systems and methods herein impact the surface realization (e.g. thechoice of words) in the natural language generation component 138 of theagent 142.

For example, in FIG. 6, input from the user 140 generates understandingoutput from the understanding model in item 130, using semantic parserand dialog act recognizer elements. The understanding model 130 feedsthe output into the dialogue manager 132, which can include exemplarymodules Otto, Optimus, Otto v2, etc., and which provides output to thegeneration model 138 (e.g., using a SimpleNLG model and generation rulesidentifier). A knowledge base is also used, as shown in item 134, whichincludes a semantic enrichment engine and predictive queries. A customermodel 136 is also used which includes a skills identifier. The customeragent 142 in FIG. 6 can also use various tools such as an apprenticemodule, annotation server, and dialog explorer.

Thus, the systems and methods herein provide the ability to understandcustomers at a psychological level, and can be utilized in a number ofways on various social media platforms. For example, the systems andmethods herein can be used in outward engagement and can help understandwhich customers are most likely open to receiving a targeted marketingcampaign. The systems and methods herein also can be used to determinewhen a targeted marketing campaign would be appropriate based on mood ofthe customer. At the same time, the systems and methods are able topersonalize the campaign in such a way that it resonates in the best waywith different types of customers. The systems and methods herein alsocan be used to provide personalized product/service recommendations.

FIG. 7 is flowchart illustrating exemplary methods herein. In item 150,these methods automatically monitor text and/or speech communicationsbetween users (e.g., using a specialized language processor). Thus, thecommunications between the users that is monitored includes, but is notlimited to, evidence that can be extracted from actual text of a dialog,speech signal, or features from a speech signal of a given dialog, etc.Such methods then automatically analyze the text communications usingthe specialized language processor to simultaneously determine, for acurrent time period, mental state variables of the user, as shown initem 152. These mental state variables can include the emotion, mood,and personality of the user.

These methods then automatically aggregate the emotion, mood, andpersonality using a hierarchical probabilistic graphical model (e.g., adirected probabilistic graph (DPG)) as shown in item 154. Whenaggregating the emotion, mood, and personality in item 154, thesemethods can, for example, maintain a single state for personality forthe time period, and can maintain multiple states for the emotion andthe mood for the time period as shown in item. Thus, if personality isknown accurately, it is just one value across the interaction. However,if there is no prior knowledge of personality, and is made as a decisionat one point in the dialogue, the methods herein may revise this valueat a later stage. This does not however mean multiple personality nodes,it means the first value for the node was incorrect, so it wasoverwritten.

For example, the directed probabilistic graph can include a singlepersonality node, multiple mood nodes, multiple emotion nodes, andmultiple evidence nodes. Each of the mood nodes, the emotion nodes, andthe evidence nodes can be for a different time portion of the timeperiod. The evidence nodes can include different dialogue variables usedby the personality node, the mood nodes, and the emotion nodes. Theemotion nodes can be, for example, happy-for-satisfaction, anger, ordistressed states; the mood nodes can be, for example, positive,neutral, or negative; and the personality nodes can be, for example,neuroticism, extraversion, openness to experience, agreeableness, orconscientiousness.

The directed probabilistic graph has edges connecting the personalitynode, the mood nodes, the emotion nodes, and the evidence nodes; and theedges themselves have probability values. Therefore, as shown in item156, these methods also determine the highest probability path throughthe directed probabilistic graph to infer the mental state of the user.When processing the paths through the directed probabilistic graph initem 156, these methods aggregate the probabilities of the edges of theseries of adjacent nodes, and the highest probability path is the paththat has an aggregation of the probabilities of the edges that is higherthan all other possible paths through the directed probabilistic graph.

As seen in item 158, using the specialized language processor, themethods output the mental state of the user from the specializedlanguage processor by displaying the emotion, mood, and personality onthe graphic user interface of the processor, or by providing the mentalstate to a separate process, such as a virtual agent. As shown in item160, these methods can also outputs any change in the variable mentalstate of the user as the conversation progresses.

The hardware described herein plays a significant part in permitting theforegoing methods to be performed, rather than function solely as amechanism for permitting a solution to be achieved more quickly, (i.e.,through the utilization of a computer for performing calculations). Aswould be understood by one ordinarily skilled in the art, the processesdescribed herein cannot be performed by human alone (or one operatingwith a pen and a pad of paper) and instead such processes can only beperformed by a machine. Specifically, processes such as automaticallymonitoring text communications between users using a specializedlanguage processor, automatically analyzing the text communicationsusing the specialized language processor to simultaneously determine,for a current time period, mental state variables of a user,automatically aggregating the emotion, mood, and personality using ahierarchical probabilistic graphical model to determine the highestprobability path through a directed probabilistic graph to infer themental state of the user use different specialized machines and cannotbe performed by humans alone.

Additionally, the methods herein solve many highly complex technologicalproblems. For example, as mentioned above, it is difficult for automatedor real customer service agents to know the mental state of theindividual with which they are conducting a text chat. Therefore, thesystems and methods herein provide the ability determine the mentalstate of a user and display the mental state or output the mental stateto another process, such as a virtual agent.

As shown in FIG. 8, exemplary systems and methods herein include variouscomputerized devices 200, 204 located at various different physicallocations 206. The computerized devices 200, 204 can include printservers, printing devices, personal computers, etc., and are incommunication (operatively connected to one another) by way of a localor wide area (wired or wireless) network 202.

FIG. 9 illustrates a computerized device 200, which can be used withsystems and methods herein and can comprise, for example, a printserver, a personal computer, a portable computing device, etc. Thecomputerized device 200 includes a controller/tangible processor 216 anda communications port (input/output) 214 operatively connected to thetangible processor 216 and to the computerized network 202 external tothe computerized device 200. Also, the computerized device 200 caninclude at least one accessory functional component, such as a graphicaluser interface (GUI) assembly 212. The user may receive messages,instructions, and menu options from, and enter instructions through, thegraphical user interface or control panel 212.

The input/output device 214 is used for communications to and from thecomputerized device 200 and comprises a wired device or wireless device(of any form, whether currently known or developed in the future). Thetangible processor 216 controls the various actions of the computerizeddevice. A non-transitory, tangible, computer storage medium device 210(which can be optical, magnetic, capacitor based, etc., and is differentfrom a transitory signal) is readable by the tangible processor 216 andstores instructions that the tangible processor 216 executes to allowthe computerized device to perform its various functions, such as thosedescribed herein. Thus, as shown in Figure ?+2, a body housing has oneor more functional components that operate on power supplied from analternating current (AC) source 220 by the power supply 218. The powersupply 218 can comprise a common power conversion unit, power storageelement (e.g., a battery, etc), etc.

While some exemplary structures are illustrated in the attacheddrawings, those ordinarily skilled in the art would understand that thedrawings are simplified schematic illustrations and that the claimspresented below encompass many more features that are not illustrated(or potentially many less) but that are commonly utilized with suchdevices and systems. Therefore, Applicants do not intend for the claimspresented below to be limited by the attached drawings, but instead theattached drawings are merely provided to illustrate a few ways in whichthe claimed features can be implemented.

Many computerized devices are discussed above. Computerized devices thatinclude chip-based central processing units (CPU's), input/outputdevices (including graphic user interfaces (GUI), memories, comparators,tangible processors, etc.) are well-known and readily available devicesproduced by manufacturers such as Dell Computers, Round Rock Tex., USAand Apple Computer Co., Cupertino Calif., USA. Such computerized devicescommonly include input/output devices, power supplies, tangibleprocessors, electronic storage memories, wiring, etc., the details ofwhich are omitted herefrom to allow the reader to focus on the salientaspects of the systems and methods described herein. Similarly,printers, copiers, scanners and other similar peripheral equipment areavailable from Xerox Corporation, Norwalk, Conn., USA and the details ofsuch devices are not discussed herein for purposes of brevity and readerfocus.

In addition, terms such as “right”, “left”, “vertical”, “horizontal”,“top”, “bottom”, “upper”, “lower”, “under”, “below”, “underlying”,“over”, “overlying”, “parallel”, “perpendicular”, etc., used herein areunderstood to be relative locations as they are oriented and illustratedin the drawings (unless otherwise indicated). Terms such as “touching”,“on”, “in direct contact”, “abutting”, “directly adjacent to”, etc.,mean that at least one element physically contacts another element(without other elements separating the described elements). Further, theterms automated or automatically mean that once a process is started (bya machine or a user), one or more machines perform the process withoutfurther input from any user. It will be appreciated that theabove-disclosed and other features and functions, or alternativesthereof, may be desirably combined into many other different systems orapplications. Various presently unforeseen or unanticipatedalternatives, modifications, variations, or improvements therein may besubsequently made by those skilled in the art which are also intended tobe encompassed by the following claims. Unless specifically defined in aspecific claim itself, steps or components of the systems and methodsherein cannot be implied or imported from any above example aslimitations to any particular order, number, position, size, shape,angle, color, or material.

What is claimed is:
 1. A method comprising: automatically monitoringcommunications between users using a specialized language processor;automatically analyzing said communications using said specializedlanguage processor to simultaneously determine, for a current timeperiod, mental state variables of a user of said users, said mentalstate variables comprising: an emotion of said user; a mood of saiduser; and a personality of said user; and automatically aggregating saidemotion, said mood, and said personality using a hierarchicalprobabilistic graphical model that determines a highest probability paththrough a directed probabilistic graph to infer the mental state of saiduser, using said specialized language processor; outputting said mentalstate of said user from said specialized language processor, saiddirected probabilistic graph maintaining a single state for saidpersonality for said time period, and maintaining multiple states forsaid emotion and said mood for said time period.
 2. The method accordingto claim 1, said directed probabilistic graph comprising a singlepersonality node, multiple mood nodes, multiple emotion nodes, andmultiple evidence nodes.
 3. The method according to claim 2, saiddirected probabilistic graph comprising edges connecting saidpersonality node, said mood nodes, said emotion nodes, and said evidencenodes, and said edges comprising probability values.
 4. The methodaccording to claim 3, a path through said directed probabilistic graphcomprising a series of adjacent nodes, a probability of said pathcomprising an aggregation of said probabilities of said edges of saidseries of adjacent nodes, and said highest probability path having anaggregation of said probabilities of said edges that is higher than allother possible paths through said directed probabilistic graph.
 5. Themethod according to claim 2, each of said mood nodes, said emotionnodes, and said evidence nodes being for a different time portion ofsaid time period.
 6. The method according to claim 2, said evidencenodes comprising different dialogue variables used by said personalitynode, said mood nodes, and said emotion nodes.
 7. The method accordingto claim 1, said emotion comprising happy-for-satisfaction, anger, ordistress, said mood comprising positive, neutral, or negative, and saidpersonality comprising neuroticism, extraversion, openness toexperience, agreeableness, or conscientiousness.
 8. A method comprising:automatically monitoring text communications between users using aspecialized language processor; automatically analyzing said textcommunications using said specialized language processor tosimultaneously determine, for a current time period, mental statevariables of a user of said users, said mental state variablescomprising: an emotion of said user; a mood of said user; and apersonality of said user; and automatically aggregating said emotion,said mood, and said personality using a hierarchical probabilisticgraphical model that determines a highest probability path through adirected probabilistic graph to infer the mental state of said user,using said specialized language processor; outputting said mental stateof said user from said specialized language processor by displaying saidemotion, said mood, and said personality on a graphic user interfaceoperatively connected to said specialized language processor, saiddirected probabilistic graph maintaining a single state for saidpersonality for said time period, and maintaining multiple states forsaid emotion and said mood for said time period.
 9. The method accordingto claim 8, said directed probabilistic graph comprising a singlepersonality node, multiple mood nodes, multiple emotion nodes, andmultiple evidence nodes.
 10. The method according to claim 9, saiddirected probabilistic graph comprising edges connecting saidpersonality node, said mood nodes, said emotion nodes, and said evidencenodes, and said edges comprising probability values.
 11. The methodaccording to claim 10, a path through said directed probabilistic graphcomprising a series of adjacent nodes, a probability of said pathcomprising an aggregation of said probabilities of said edges of saidseries of adjacent nodes, and said highest probability path having anaggregation of said probabilities of said edges that is higher than allother possible paths through said directed probabilistic graph.
 12. Themethod according to claim 9, each of said mood nodes, said emotionnodes, and said evidence nodes being for a different time portion ofsaid time period.
 13. The method according to claim 9, said evidencenodes comprising different dialogue variables used by said personalitynode, said mood nodes, and said emotion nodes.
 14. The method accordingto claim 8, said emotion comprising happy-for-satisfaction, anger, ordistress, said mood comprising positive, neutral, or negative, and saidpersonality comprising neuroticism, extraversion, openness toexperience, agreeableness, or conscientiousness.
 15. A systemcomprising: a specialized language processor; and a graphic userinterface operatively connected to said specialized language processor,said specialized language processor automatically monitoringcommunications between users, said specialized language processorautomatically analyzing said communications to simultaneously determine,for a current time period, mental state variables of a user of saidusers, said mental state variables comprising: an emotion of said user;a mood of said user; and a personality of said user, said specializedlanguage processor automatically aggregating said emotion, said mood,and said personality using a hierarchical probabilistic graphical modelthat determines a highest probability path through a directedprobabilistic graph to infer the mental state of said user, said graphicuser interface outputting said mental state of said user from saidspecialized language processor by displaying said emotion, said mood,and said personality, and said directed probabilistic graph maintaininga single state for said personality for said time period, andmaintaining multiple states for said emotion and said mood for said timeperiod.
 16. The system according to claim 15, said directedprobabilistic graph comprising a single personality node, multiple moodnodes, multiple emotion nodes, and multiple evidence nodes.
 17. Thesystem according to claim 16, said directed probabilistic graphcomprising edges connecting said personality node, said mood nodes, saidemotion nodes, and said evidence nodes, and said edges comprisingprobability values.
 18. The system according to claim 17, a path throughsaid directed probabilistic graph comprising a series of adjacent nodes,a probability of said path comprising an aggregation of saidprobabilities of said edges of said series of adjacent nodes, and saidhighest probability path having an aggregation of said probabilities ofsaid edges that is higher than all other possible paths through saiddirected probabilistic graph.
 19. The system according to claim 16, eachof said mood nodes, said emotion nodes, and said evidence nodes beingfor a different time portion of said time period, said evidence nodescomprising different dialogue variables used by said personality node,said mood nodes, and said emotion nodes.
 20. The system according toclaim 15, said emotion comprising happy-for-satisfaction, anger, ordistress, said mood comprising positive, neutral, or negative, and saidpersonality comprising neuroticism, extraversion, openness toexperience, agreeableness, or conscientiousness.